An automated deep learning pipeline for EMVI classification and response prediction of rectal cancer using baseline MRI: a multi-centre study

The classification of extramural vascular invasion status using baseline magnetic resonance imaging in rectal cancer has gained significant attention as it is an important prognostic marker. Also, the accurate prediction of patients achieving complete response with primary staging MRI assists clinicians in determining subsequent treatment plans. Most studies utilised radiomics-based methods, requiring manually annotated segmentation and handcrafted features, which tend to generalise poorly. We retrospectively collected 509 patients from 9 centres, and proposed a fully automated pipeline for EMVI status classification and CR prediction with diffusion weighted imaging and T2-weighted imaging. We applied nnUNet, a self-configuring deep learning model, for tumour segmentation and employed learned multiple-level image features to train classification models, named MLNet. This ensures a more comprehensive representation of the tumour features, in terms of both fine-grained detail and global context. On external validation, MLNet, yielding similar AUCs as internal validation, outperformed 3D ResNet10, a deep neural network with ten layers designed for analysing spatiotemporal data, in both CR and EMVI tasks. For CR prediction, MLNet showed better results than the current state-of-the-art model using imaging and clinical features in the same external cohort. Our study demonstrated that incorporating multi-level image representations learned by a deep learning based tumour segmentation model on primary MRI improves the results of EMVI classification and CR prediction with good generalisation to external data. We observed variations in the contributions of individual feature maps to different classification tasks. This pipeline has the potential to be applied in clinical settings, particularly for EMVI classification.

Reporting on race, ethnicity, or other socially relevant groupings

Ethics oversight
Note that full information on the approval of the study protocol must also be provided in the manuscript.

Field-specific reporting
Please select the one below that is the best fit for your research.If you are not sure, read the appropriate sections before making your selection.

Life sciences
Behavioural & social sciences Ecological, evolutionary & environmental sciences For a reference copy of the document with all sections, see nature.com/documents/nr-reporting-summary-flat.pdf

Life sciences study design
All studies must disclose on these points even when the disclosure is negative.

Blinding
Behavioural & social sciences study design All studies must disclose on these points even when the disclosure is negative.

Research sample
Sampling strategy

Data exclusions
Non-participation

Randomization
Primary staging MRIs (T2W and DWI) of 509 rectal cancer patients treanted with neoadjuvant chemoraodiotherapy were collected from 9 medical centers.
Specific cases were removed in the study, including predominately mucinous tumour type, multiple tumours in field-of-view, tumours not competely covered in field of view non-diagnostic quality, severe geometric mismatch between t2w and dwi, prolonged waiting interval between chemoradiotheraly and surgery (>20 weeks) and concomitant abscesses Models were verified and replicated using regular machine learning metrics on independent test cohort.
The enrolled samples were randomly divided into the training cohort and test cohort center-wise for multi-centre study and samples were randomly divided into the training cohort and test cohort for single center analysis.
The investigator were blinded to group allocation during data collection and analysis.

Ecological, evolutionary & environmental sciences study design
All studies must disclose on these points even when the disclosure is negative.Confirm that both raw and final processed data have been deposited in a public database such as GEO.
Confirm that you have deposited or provided access to graph files (e.g.BED files) for the called peaks.

Data access links
May remain private before publication.

Files in database submission
Genome browser session The axis labels state the marker and fluorochrome used (e.g.CD4-FITC).
The axis scales are clearly visible.Include numbers along axes only for bottom left plot of group (a 'group' is an analysis of identical markers).
All plots are contour plots with outliers or pseudocolor plots.
A numerical value for number of cells or percentage (with statistics) is provided.The study design for the proposed project is a retrospective, multi-center cohort study

T2 Weighted Imaging and Diffusion Weighted Imaging
The study aim to retrospectively analyze the routinely acquired primary staging imaging (MRI) of patients treated with neoadjuvant for rectal cancer from 2012 to 2018 1.5T, 3.0T nnUNET self-configuring normalization scheme,0.5 and 99.5 percentiles of the foreground voxels for clipping as well as the global foreground mean and s.d. for the normalization of all images.

Data with non-diagnostic quality was excluded
Not a longitutial study pelvis and specifically focuses on the rectum and its surrounding structures.

Functional
predictive analysis This checklist template is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material.If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/Dice similarity score (Dice), Area under the Curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and F1 score Deep learning models including one rectal tumour segmentation network and one classfication network for EMVI classfication and treatment response